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ABSTRACT 


The  Cramer-Rao  inequality  is  used  to  determine  a  lower  bound  on  the  variance  with  which 
a  sinusoidal  frequency  can  be  estimated  in  the  presence  of  Gaussian  white  noise.  A 
parametric  study  has  elucidated  the  influence  of  number  of  samples  (N),  sampling 
frequency  (1/A),  phase  (</>),  and  signal-to-noise  ratio  (SNR)  on  the  Cramer-Rao  bound.  A 
closed  form  expression  for  the  asymptotic  level  to  which  the  Cramer-Rao  bound  decays  is 
characterized  and,  for  low  frequencies,  the  bound  is  determined  analytically  and 
graphically.  The  form  of  the  Cramer-Rao  bound  is  linked  to  resolution  in  the  sampling 
problem.  Identification  of  trade-offs  characterizing  the  sensitivity  of  the  bound  and 
parameters  associated  with  it  are  discussed. 
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CRAMfeR-RAO  BOUND  ANALYSIS  FOR  FREQUENCY 
ESTIMATION  OF  SINUSOIDS  IN  NOISE 

I.  INTRODUCTION 


A  problem  of  interest  is  estimation  of  the  frequency  of  a  sinusoidal  oscillation  from  noisy 
sampled  data.  The  particular  emphasis  in  this  report  is  on  the  case  in  which  only  a  fraction  of  a 
cycle  is  sampled.  It  is  hoped  that  the  guidance  achieved  in  solving  this  problem  could  be  applied 
to  the  more  complex  problem  of  motion  other  than  a  pure  sinusoid.  The  approach  here  is  to  use 
the  Cramer-Rao  bound  as  a  calculation  tool.  The  Cramer-Rao  bound  is  a  lower  bound  on  the 
variance  in  parameter  estimation  in  the  case  when  the  estimator  is  unbiased. 

The  problem  of  frequency  estimation  for  single  and  multiple  complex  sinusoids  in  noise  has 
received  attention  in  the  literature  [1,2]  for  the  case  of  frequency  large  compared  with  the 
reciprocal  of  the  measurement  interval  (1/NA).  A  maximum  likelihood  method  for  frequency 
estimation  of  real  sinusoids  in  noise,  with  improved  computation  efficiency,  has  already  been 
presented  [3].  Of  interest  to  the  problem  of  determination  of  frequencies  is  the  case  in  which  the 
observation  period  NA  (N  =  number  of  samples,  A  =  sampling  interval)  is  short  compared  with 
the  oscillation  period.  In  this  case  a  good  signal-to-noise  ratio  (SNR)  can  compensate  for  the 
short  measurement  interval. 

Examination  of  the  Cramer-Rao  bound  follows  its  derivation  from  the  original  problem  ol 
sampling  a  sine  wave.  A  frequency  independent  form  for  the  Cramer-Rao  bound  is  presented 
which  is  accurate  for  frequencies  between  those  designated  by  the  Nyquist  criterion.  The 
frequency  dependence  of  the  bound  at  low  frequencies,  for  initial  phases  of  0  and  7r/2,  is 
obtained.  Finally,  the  family  of  Cramer-Rao  bound  curves  parameterized  by  N  and  A  is  reduced 
to  a  single  operating  curve  from  which  trade-offs  involving  NA  and  SNR,  as  well  as  an 
expression  for  the  bound  for  low  frequencies,  are  determined. 


II.  DERIVATION  OF  THE  CRAMfeR-RAO  BOUND 


The  general  problem  of  estimation  of  a  sinusoidal  frequency  in  Gaussian  white  noise  derives 
from  taking  N  samples  of  the  system  separated  by  successive  sampling  intervals  A 

,  Sj  =  A  sin  (27rfAi  +  <j>)  , 


Y,  =  Sj  +  Wj 


(i  =  1  .  .  .  N) 


(1) 


where  A,  f,  and  </>  are  oscillation  amplitude,  frequency,  and  phase.  The  measurement  errors  W, 

are  Gaussian  and  independently  distributed  with  zero  mean  and  variance  oR.  Since  the 

(Y,  Sj)  =  W;  are  therefore  also  Gaussian,  the  probability  density  function  for  these  terms  can  be 

represented: 


P(Y/S)  = 


1 


1 


N 


-V  2  (Yi  -  S.)2/4 

1  i=  I 


(27T)N/2ffN  exP 

The  Cramer-Rao  inequality  is  a  lower  bound  on  the  variance  in  parameter  estimation  and  is 


(2) 


In  P(Y/S 
as 


d  In  P( Y/S)  \  2 
da 


A' 

where  a  = 

f 

0 

(3) 


is  the  log  likelihood  function  which  can  be  discerned  from  the  above  density  function 


N 


In  P(Y  /  S)  =  1/2  X  (Yi-Sj)2/a2R 
i=l 

ignoring  the  constant  terms.  The  derivative  of  the  log  likelihood  function  with  respect  to  the 
parameters  of  the  sinusoid. 


(4) 


a  - 


is 


d  In  P(Y/S) 
da 


N 

=  2  (Yi 

i=l 


Sj) 


dSL 

da 


(5) 


Squaring  this  expression  and  taking  its  expectation  results  in: 


/ d  In  P(Y/S) V 

-  C 

N  N  (Yj  -  Sj)  (Yj  Sj)  dSj  dSj j 

•te  / 

-  C 

i=l  j“l  4  da  d0f 

(6) 


2 


E[(Yj  -  S,)  (Yj  -  Sj)]  =  0  except  at  i  =  j  where  it  equals  ct^. 


Thus  E 


d  In  P(Y/S) 


dS: 

where  — ; —  = 
da 


da 

dS: 


2  N 

=  2 

i=l 


dSj 

da 


l 

da 


(7) 


<9A 

dS; 


df 

dS; 


d<fr 

and  each  element  (jk)  of  the  resulting  Fisher  information  matrix 


N 


B  equals  V 


1 


<9S,  dS : 


i=l 


(9aj 


dat 


It  is  the  inverse  of  B  which  yields  the  Cramer-Rao  bound  on  estimation  of  the  a,  The 
Cramer-Rao  bound  can  be  stated  as 

Var  (ai  -  aJ)  ^  BU 

where  a,  is  an  unbiased  estimate  of  aj  and  B-U  is  the  jth  diagonal  element  of  the  inverse  of  B. 
The  matrix  B  to  be  inverted  is  symmetric  and  is  shown  explicitly  below: 


N 

b„=  2 

i=l 


1  2 

—  (sin  (2?rfAi  +  <f>)) 


N  ,  2 

®22  =  2  — 2~  (2?rAiA  cos  (27rfAi  +  <p)) 

i=  1  °R 


N  j  2 

B33  =  2)  — 2  cos  (2rrfAi  +  $)) 

1=1  aR 


(8) 


N 


B,2  =  B2,  =  2)  — 2~  (2n-AiA  sin  (27rfAi  +  <f>)  cos  (2n-fAi  +  </>)) 


i=l  aR 


N  j 

B ) 3  =  B31  =  ^  — 2 —  (A  sin  (27rfAi  +  </>)  cos  (27rfAi  +  <p)) 

i=l  °R 


N  j  2 

B23  =  B32  =  ^  — —  (27rAi  (A  cos  (27rfAi  +  </>))  ) 

i=l  aR 


3 


Rather  than  working  with  the  Cramer-Rao  bound  in  terms  of  variance,  the  root  mean 
square  error  in  frequency  estimation,  (B22)1/2,  will  be  used  here  whenever  the  Cramer-Rao  bound 
is  referred  to  subsequently.  (B11)1/2  and  (B33)’/2  are  the  Cramer-Rao  bounds  for  estimation  of 
amplitude  and  phase,  respectively.  (A/ctr)  is  the  square  root  of  the  signal-to-noise  ratio  (V SNR) 
and  is  taken  to  be  equal  to  1  everywhere  except  in  Section  VI. 
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III.  NUMERICAL  RESULTS:  PHASE  (<*>)  =  0 


One  of  the  first  eharaeteristies  of  the  Cramer-Rao  bound  with  whieh  to  be  eoneerned  is  its 
behavior  at  the  Nyquist  frequency  1/A  =  2f.  This  corresponds  to  the  ease  in  which  the  sinusoidal 
frequency  is  sampled  exactly  at  its  nodes.  Since  frequency  estimation  is  then  impossible,  the 
Cramer-Rao  bound  is  infinite.  For  f  =  k/2A  (where  k  =  1,  2  .  .  .  )  the  infinity  repeats  for  higher 
frequencies.  Figure  I  shows  the  behavior  of  the  Cramer-Rao  bound  for  the  sampling  parameters 
N  =  16,  A  =  0.008.  The  Cramer-Rao  bound  (B22)1  2  is  computed  from  (8)  and  is  singular  at 
1  =  1  / 2 A  62.5  and  all  multiples. 


S  0.0  62  5  125  0 

*  FREQUENCY  (Hz) 

f  igure  l.  Xyquisi  intervals  of  Cramer-Rao  bound.  V=  16.  A  -  0.008,  $>  =  (). 

1  he  following  figure  exhibits  the  Cramer-Rao  bound  behavior  for  frequencies  much  less  than 
the  Nyquist  frequency.  The  Cramer-Rao  bound  (B22)1  2  computed  from  (8)  is  plotted  in  Figure  2, 
taking  N  =  16,  32,  64,  128,  256  and  512  successive  samples  separated  in  time  by  a  constant 
interval  of  A  =  0.008  s.  It  is  immediately  obvious  that  the  Cramer-Rao  bound  decreases  when  N, 
the  number  of  samples  taken,  increases.  1  he  break  point  between  the  monotome  and  oscillators 
parts  of  each  curve  occurs  at  f  =  1  2NA  whieh  is  at  increasingly  lower  frequency  as  N  becomes 
greater.  I  he  third  trend  is  that  the  Cramer-Rao  bound  decays  to  an  asymptotie  level  as  frequency 
increases.  \/SNR  has  been  taken  to  be  1  here,  however,  it  is  implicit  from  Equations  (8)  for  B22 
that  the  Cramer-Rao  bound  shown  in  Figures  1  and  2  should  be  divided  by  \J SNR  in  the  more 
general  ease. 
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Hgure  2.  Cramer-  Rao  hound  for  frequency  estimation.  A  =0.008,  </>  =  0. 

Hgure  3  exhibits  the  Cramer-Rao  hound  for  NA  =  1.  The  family  of  curves  consists  of  the 
eases  in  which  N  =  16.  32,  64,  128,  256  and  512  are  matched  with  A  =  0.0625,  0.0313,  0.0156. 
0.0078,  0.0039,  and  0.002  respectively.  The  break  point  is  now  at  approximately 
1  ~  1  2\A  1:3  0.5  Hz  lor  all  six  curves.  For  the  portions  of  curves  shown  in  Figure  3,  the 
oscillatory  parts  ol  the  bounds  are  identical  except  for  an  offset  in  asymptotic  level. 

On  account  ol  the  Nyquist  criterion,  for  which  sampling  is  only  at  nodes  of  a  sinusoid, 
infinities  repeat  lor  1  =  I  2A  (k  =  1,  2  .  .  .).  For  the  ease  in  Figure  2  these  infinities  repeated  at 
the  same  frequency  lor  all  curves.  Each  ol  the  curves  in  Figure  3,  however,  has  its  own  period 
imposed  by  the  Nyquist  criterion  as  A  is  different  for  each.  Thus  the  N  =  16  curve  in  Figure  3 
will  give  rise  to  a  singularity  at  f  «*  8,  the  N  =  32  curve  at  f «  16,  and  so  on.  Our  primary 
interest,  however,  is  in  the  low*  frequency  portions  of  these  curves. 
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Figure  J  Cramer- Kao  bound  /or  frequency  csiinunion.  <t>  =  0.  -IN  =  / 
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IV.  DERIVATION  OF  THE  ASYMPTOTIC 
CRAMfiR-RAO  BOUND  (<J>  =  0) 


The  analytical  problem  is  to  simplify  the  summations  which  form  the  terms  of  the  Fisher 
information  matrix.  It  is  of  interest  to  examine  the  case  in  which  trigonometric  oscillatory  terms 
in  the  Fisher  information  matrix,  which  depend  on  frequency,  are  summed  [6]: 


B,  |  =  N/2 
B22  “  (2t tA)2 

B33  =  N/2 
Bj2  =  B2|  =  o 
Bi3=B3l  =  0 

B23  =  Bn  =  (2nA)  \-tf-  +  -f-J  (9) 

It  has  been  observed  that  the  asymptotic  levels  of  the  Cramer-Rao  bound  (B 22)l/2,  in  the 
periodically  decaying  region  between  frequencies  designated  by  the  Nyquist  criterion,  can  be 
calculated  according  to  (9)  as  functions  of  N  and  A. 

For  Figures  2  and  3  this  region  of  frequencies  falls  to  the  right  of  the  first  break  point 
where  f  >  1/2NA.  This  is  equivalent  to  the  condition  that  half  the  period  T/2  is  exceeded  by  the 
observation  interval  NA.  The  Cramer-Rao  bound  can  be  approximated  from  (9)  for  frequencies 
to  the  right  of  the  break  points  where  the  bound  levels  off,  however,  to  only  as  far  as  I/2NA  less 
than  the  first  f  =  1/2A  Nyquist  infinity.  Subsequently,  the  Cramer-Rao  bound  can  be  obtained  in 
this  manner  for  respective  frequencies  in  the  higher  order  Nyquist  periods. 


/  N3 
\6 


N3  .  N2 


M2 


M 


Solving  for  (B22)1^  the  asymptotic  form  for  the  Cramer-Rao  bound  for  frequencies  in  the 
periodically  decaying  region  is  obtained: 


(B22)«/2 


— — ) 

N(N2  -  10)  f 


1/2 


(10) 


Substituting  for  N  and  A,  it  is  found  that  the  values  of  this  expression  equal  almost  precisely  the 
frequency  independent  asymptotic  levels  for  the  Cramer-Rao  bounds  which  are  shown  partially  in 
Figures  2  and  3.  In  the  following  table  (Table  1)  are  values  of  (B22)1/2  which  were  computed 
from  (9)  alongside  those  derived  from  the  full  Fisher  information  matrix  (8). 


Substitution  of  the  sine  and  cosine  series  representations  for  small  arguments  into 
Equations  (8)  (<£  =  0)  has  also  been  achieved.  This  has  determined  that  the  asymptotic  frequency 
dependence  of  the  Cramer-Rao  bound  at  low  frequencies  is  proportional  to  1  /  f 2  for  <p  -  0. 
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TABLE  1 

Calculated  Asymptotic  Cramer-Rao  Bound 

N 

A 

(B22)1/2 

from  (9) 

from  (10) 

16 

0.008 

1.61 

1.55 

32 

0  008 

0.55 

0.54 

64 

0.008 

0.19 

0.19 

128 

0  008 

0.068 

0.068 

256 

0.008 

0.024 

0.024 

512 

0008 

0.008 

0.008 

16 

0.0625 

0.206 

0.199 

32 

0.0313 

0.141 

0.138 

64 

0.0156 

0099 

0.098 

128 

0.0078 

0.070 

0.069 

256 

0.0039 

0049 

0  049 

512 

0.002 

0.034 

0  034 
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V.  NUMERICAL  RESULTS  FOR  NON-ZERO  PHASE 


Figure  4  depicts  the  Cramer-Rao  bound  as  a  function  of  frequency  when  </>  =  0,  7t/4,  tt/2 
and  3tt/4  for  the  case  in  which  N  =  64  and  A  =0.008.  The  bound  was  computed  from  the  full 
Fisher  information  matrix  (8).  It  can  be  discerned  that  the  oscillations  decay  so  that 
asympotically  the  curves  all  approach  a  single  Cramer-Rao  bound  level  equal  to  that  for  </>  =  0, 

N  =  64,  A  =  0.008  (Figure  2).  The  frequency  dependence  of  the  bound  at  low  frequencies,  for 
</>  =  7t/2,  is  determined  to  be  1/f.  For  frequencies  less  tha  0.2  Hz,  the  asymptotic  behavior  of  the 
Cramer-Rao  bounds  for  =  0  and  </>  =  tt/2  form  an  envelope  about  all  of  the  phases  shown  in 
Figure  4. 

F'igure  5  departs  from  Figure  3  by  only  a  small  amount  for  f  >  1.  Figure  3  was  obtained 
with  <£  =  0.  The  former  is  created  by  taking  the  Cramer-Rao  bound  [computed  from 
Equations  (8)]  with  the  phase  which  yields  the  highest  Cramer-Rao  bound  out  of  100  phases 


Figure  4.  Cramer-Rao  bound  for  frequency  estimation.  N  =  64,  A  =  0.008. 
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equally  spaced  between  0  and  tt.  The  behavior  of  the  Cramer-Rao  bound  for  worst  case  phase 
has  already  been  shown  in  [2],  however,  this  is  for  the  case  of  a  constant  sampling  interval  A 
and  large  frequency. 

NA  is  constrained  to  be  equal  to  1  for  both  Figures  3  and  5.  The  break  point  between  the 
monotonic  and  periodic  segments  in  the  family  of  curves  in  Figure  5  occurs  at  approximately 
f=  1/NA,  while  for  Figure  3  is  at  approximately  1/2NA.  The  lower  break  point  for  the  case  of 
(f>  -  0  is  equal  to  the  first  null  in  sin  Nw/sin  w,  where  w  =  7r/N,  and  w  =  27rfA,  which  is  exactly 
f=  1/2NA. 


Figure  5.  Cramer- Ran  hound  for  frequency  estimation.  Worst  ease  </>.  SS  -  l. 
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VI.  NORMALIZATION  OF  THE  CRAMER-RAO  BOUND: 
TRADE-OFFS  INVOLVING  NA  AND  SNR 


One  conclusion  which  can  be  drawn  from  Figures  2  and  3  in  the  previous  section  is  that  the 
width  of  the  main  lobe  of  the  Cramer-Rao  bound  curves  is  the  parameter  1/2NA.  This  can  be 
identified  with  the  resolution  of  the  estimation  process.  The  greater  the  observation  interval  NA, 
the  smaller  will  be  the  resolution  <5f  =  I/2NA.  This  trend  can  be  likened  to  that  of  a  square  wave 
and  its  Fourier  transform.  The  wider  the  square  wave  is,  the  smaller  the  width  to  the  first  null  of 
its  Fourier  transform  will  be. 

In  Figure  6,  the  Cramer-Rao  bound  <?f  normalized  by  the  resolution  Sf  =  I/2NA  is  plotted 
against  the  frequency  normalized  by  <5f.  This  is  achieved  for  N  =  16,  32,  64,  128,  256,  and  512,  for 
A  =  0.008  in  each  case.  These  are  the  same  parameters  as  were  used  for  the  unnormalized  case  in 
Figure  2.  The  curves  of  the  family  which  results  all  break  at  the  same  f/<5f  and  are  separated  by  a 
constant  multiple  for  all  f/<5f. 


Figure  6.  Normalized  Cramer-Rao  bound  for  frequency  estimation.  <t>  -  0 
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It  was  shown  in  Equation  (10)  of  Section  IV  that  the  asymptotic  Cramer-Rao  bound  is  on 
the  order  of  1/N3/2A.  Thus  the  normalized  Cramer-Rao  bound  0f/Sf  is  on  the  order  of  l/\/N. 
Another  piece  of  information  we  can  extract  from  scaling  the  Cramer-Rao  bound  is  that  the 
family  of  curves  in  Figure  6  can  be  collapsed  to  one  of  \/N  Of/df  versus  f/6f.  This  is  shown  in 
Figure  7  where  for  numerical  convenience  another  scaling  factor  \J  1  /64  has  been  included.  The 
family  of  curves  is  thus  represented  by  the  single  relationship  of  \/N/64  Of/8f  versus  f/6f. 

From  Figure  7  it  can  be  observed  that,  in  the  low  frequency  region  where  f<  1/2NA,  the 
slope  of  the  curve  is  approximately  -2,  for  0  =  0. 


Figure  7.  Fully  normalized  Cramer-Rao  bound. 
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This  relation  represents  the  dependence  of  the  Cramer-Rao  bound  on  frequency,  for  low 
frequencies  mapped  onto  the  limb  of  the  curve  which  lies  to  the  left  of  the  break  point. 

The  following  examples  are  designed  to  exemplify  the  trade-offs  of  NA  and  SNR  which  can 
be  determined  using  Figure  7. 

Example  1:  Given  f  =  2  Hz,  5f  =  1  Hz,  and  SNR  =  1,  what  are  N  and  A  such 
that  the  Cramer-Rao  bound  Of  equals  0.1? 

For  f/5f=  2,  it  is  shown  in  Figure  8  that  \/N/64  Of/5f  equals  approximately  0.2.  Since 
cj|75f=0.1,  N  can  be  solved  for  and  is  found  to  be  256.  It  is  also  known  that  5f=  1/2NA=  1  Hz, 
so  with  N  =  256,  A  need  be  no  larger  than  0.002. 


Figure  8.  Fully  normalized  Cramer-Rao  bound  (example  l,  2). 

For  the  second  example,  SNR  is  for  the  first  time  a  variable  parameter  and  is  no  longer  set 
equal  to  1.  It  can  be  discerned  from  Fquation  set  (8)  that  f B^2)  1  /2^  the  Cramer-Rao  bound,  will 
be  proportional  to  l/A/oR  or  l/\/SNR. 
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Example  2:  Using  Figure  7,  the  problem  here  is  to  determine  what  SNR  ratio  is  necessary 
so  that  Of=  0.2  Hz  when  f  =  2  Hz,  N  =  16,  and  A  =  0.01. 

It  is  known  that  Sf  =  I/2NA  =  1/0.32  Hz  so  that  f/5f  =  0.64.  The  design  point  for  which  \/SNR 
will  be  determined  is,  for  N  =  16,  at  \/l6/64  o{/8{.  When  a{=  0.2  Hz  and  5f  =  1/0.32  Hz,  this 
point  falls  at  0.032  on  the  vertical  axis  of  Figure  8.  However,  the  actual  point  on  the  axis  to 
which  f/5f  =  0.64  corresponds  is  approximately  0.3.  Therefore  a  \/SNR  ratio  of  ***  10  is  required 
to  reduce  the  normalized  and  scaled  value  of  the  Cramer-Rao  bound  from  **  0.3  to  0.032. 
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VII.  SUMMARY  AND  CONCLUSIONS 


The  influences  of  number  of  samples,  sampling  frequency,  phase,  and  signal-to-noise  ratio  on 
the  Cramer-Rao  bound  for  frequency  estimation  of  sine  waves  in  Gaussian  white  noise  have  been 
studied.  A  closed  form  expression  for  the  asymptotic  Cramer-Rao  bound  has  been  derived  which 
exhibits  a  1/N 3/2A  dependence.  The  Cramer-Rao  bound  normalized  by  the  resolution  and  scaled 
by  \J N,  where  N  is  the  number  of  samples,  reduces  the  family  of  curves  parameterized  by  N  and 
A  to  a  single  operating  curve.  Examples  of  trade-offs  involving  NA  and  SNR  ratio  are 
determined  by  this  composite  relationship  among  the  parameters. 

As  long  as  the  measurement  interval  NA  is  at  least  a  half  cycle  of  the  sine  wave,  the 
Cramer-Rao  bound  is  close  to  being  independent  of  frequency;  however,  in  the  instance  of  worst 
case  phase  the  measurement  interval  must  cover  an  entire  cycle.  In  the  low  frequency  region 
where  NA  is  less  than  half  a  cycle,  the  bound  on  <jf  varies  as  1/f2  for  phase  0  =  0,  and  1/f  for 
phase  0  =  7t/2.  For  example,  if  the  frequency  is  0.1  Hz,  the  value  of  Of  will  be  at  most  25  times 
larger  than  for  a  frequency  of  0.5  Hz,  with  the  same  parameters  N,  A.  If  the  signal-to-noise  is 
sufficiently  high,  it  is  still  possible  to  obtain  accurate  frequency  estimates  even  in  this  low 
frequency  region. 
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